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ABSTRACT 



This paper presents and evaluates different types of student 
assessment measures- -including discussion-generating questions, multiple 
choice questions, essay questions, and authentic proj ects- -for on-line 
courses. The paper asserts that assessment should mimic how students will be 
evaluated on the job site in the real world, meaning that they should be team 
members and problem solvers, with the technological and academic skills to 
communicate the results of their actions. The findings are based on the 
researcher's experience as the teacher of an Internet -delivered course, 

"Earth Systems Science Approach to Physical Geography." The paper introduces 
the concept of "authentic assessment," a process that requires that the 
evaluation be realistic and force the students to use knowledge to solve 
real-world problems, which are often open-ended, poorly framed, and have no 
clear-cut answers. A scoring rubric, which provides the criteria for students 
to follow in order to receive a specified grade, is presented. Finally, the 
paper lists examples of the four assessment types, along with the goals, 
advantages, and disadvantages of each. It is concluded that on-line 
assessment of small Internet -based classes can be accomplished using many of 
the same assessment tools used in face-to-face classes. (CAK) 
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ABSTRACT 

Student assessment is a huge issue in Internet delivery of classes. College instructors are 
worried about how to assess student learning in traditional classes, and the problem 
appears bigger in on-line classes. If we understand what is going to be required of our 
students after they leave us, it is easier to determine how we should evaluate them, no 
matter what the delivery method. Students today are expected to be team-working 
problem solvers, with the skills (technological and academic) to communicate the results 
of problems solved. Assessment should mimic how students will be evaluated on the job 
site in the “real world.” In other words, assessment should be authentic. In this paper I 
present several examples of on-line assessment tools I have used in my Internet-delivered 
“Earth Systems Science Approach to Physical Geography.” Students can be engaged and 
evaluated with a combination of authentic discussion generating questions, open-ended 
essay questions, and authentic projects. Even multiple choice questions can be crafted to 
encourage student learning. Authentic evaluation is a much larger workload, compared to 
traditional objective evaluation tools. 



INTRODUCTION 

Evaluation of students in web based classes is a topic sure to generate discussion. When I 
talk about on-line teaching to interested colleagues, invariably one of the first questions is 
“how do you handle on-line assessment?” This question is quickly followed by “how do 
you keep them from cheating?” My answer to the first has been that I evaluate the same 
way I have in my traditional classes - 1 use multiple choice questions, short and long 
essay questions, discussion questions, and projects. The answer to the second question is 
that I cannot, so I don’t worry about it, which is, at best, a very unsatisfactory answer for 
most. 

What is it that society, mainly in the form of employers, wants from today's educated 
person? They want people who can work as members of a team to solve problems and 
who are able to communicate [1]. Today's educated person has access to reference 
material both off and on-line, modem computers with word processing, spreadsheet, and 
specialized software, and access to the experiences and backgrounds of co-workers. As 
educators, we often require that students perform in a way that is foreign to how they will 
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eventually be performing. They will never be asked on the job to sit away from all 
reference material, forced to solve problems and communicate in isolation! 

Casper College offers several traditional freshman and sophomore courses via the 
Internet. Our classes are modest in size, ranging from 5 to 25 students each and we offer 
5 to 9 Internet courses per semester. The assessment examples in this paper are taken 
from my on-line "Earth Systems Science Approach to Physical Geography," a course I 
have taught for 21 years in the traditional classroom and for two years in the cyber 
classroom [2]. 



ASSESSMENT TOOLS 

I believe that successful assessment in on-line classes lies in making the evaluation as 
realistic as possible. Students should demonstrate familiarity with a body of knowledge, 
be able to solve realistic problems, work efficiently and comfortably with each other, and 
communicate their results clearly. Essay exams, discussion questions, and projects 
require background knowledge, problem solving skills, and communications abilities. 
Doing these types of assessments on-line only adds to the realism of the assessment. I 
have evaluated student learning in my face-to-face classes using these tools; they are 
traditional evaluation tools, and I believe that I can successfully adapt them for on-line 
delivery. I feel it is possible to write materials that can fairly evaluate a student's grasp of 
the topic, their ability to problem solve, and their ability to communicate. 

The key is writing assessment materials that provoke thought rather than require simple 
memorization. Writing multiple choice questions that do not require a single, simple fact 
item for an answer takes extra time. Open-ended essay questions are relatively simple to 
write, but take much more time to evaluate. Large, complicated, and realistic projects are 
difficult to craft and time consuming to evaluate. It becomes then not a problem of how 
do we evaluate on-line work, or how do we preventing cheating, but how do we design 
assessment tools for use in small-scale classes and programs that are realistic enough to 
make cheating next to impossible. We need to work with small enough groups of students 
to make this possible and effective. 

It is too early in my own on-line experiences to be able to statistically judge the success 
of my approach. It may be that for small-scale programs such as those at many 
community colleges it will never be possible to statistically demonstrate the success of 
any particular approach to assessment. In my case, I can only base the degree of 
effectiveness on my years of teaching experience. 




AUTHENTIC ASSESSMENT 

My first encounter with the term "authentic assessment" was at a small conference on 
Internet teaching. One of the guest speakers used the term in passing, and it intrigued me. 
Each profession has many "jargon" terms, and I was curious if I could take this one at 
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face value or if it had a deeper meaning, known only to the initiated. Like many teaching 
at the college level in a content-specific area, I have had minimal exposure to the 
academic field of education. I was gratified to learn that not only does "authentic 
assessment" mean what it says, I have been doing it most of my career. ) 

According to the Educational Resources Information Center (ERIC) thesaurus[3], 

"..."Authentic assessment" is a particular context for alternative assessment in 
that authentic assessments explicitly simulate real-world problem-solving 
situations. Alternative assessment and authentic assessment both belong to the 
broader class called "Performance Based Assessment" because they both describe 
methods of student evaluation which are based upon the students' own constructed 
responses and other creations." 

Wiggins [4] gives an additional definition: 

"Assessment is authentic when we directly examine student performance on 
worthy intellectual tasks. Traditional assessment, by contract, relies on indirect or 
proxy 'items'— efficient, simplistic substitutes from which we think valid 
inferences can be made about the student's performance at those valued 
challenges." 

These definitions, and in particular the rest of Wiggins' paper, stress that the assessment 
is authentic if it is realistic and requires that students use knowledge. Real-world 
problems are sometimes open ended, poorly stated, and have no clear-cut answer. At 
times it takes a team approach to solve such problems, and the solutions may require 
specialized knowledge that is not immediately at hand in physical or human resources. If 
students are required to research, write, and communicate using the Internet, they are also 
practicing a skill that will become more important in the future. 




SCORING RUBRICS 

Students are often concerned with how grades are assigned, especially on written 
assessments. I evaluate each written assignment carefully using criteria (listed below) 
based on a paper by Tewksbury [5]. I caution students to study these criteria carefully, 
especially if they get lower grades than they think they should. On-line students in 
particular, in my experience, need to be reminded on occasion that simply putting in time 
and doing all of the work will not guarantee a high grade. I use pluses and minuses for 
work close to the borders of the next highest or next lowest grade. This scoring rubric is 
certainly not as cut and dried as an objective true- false, multiple-choice type of exam, but 
I find my scoring is remarkably internally consistent when I use it. 

CRITERIA FOR "A" RANGE GRADE: Outstanding explanation with superior 
supporting information; unusual insights and flashes of brilliance in the 
discussion; creative and original analyses and thoughts, goes well beyond the 
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minimum required for the assignment. Writing has a real sense of style, with 
excellent organization, very few grammatical or spelling errors. DOES NOT read 
like a first draft. 

CRITERIA FOR "B" RANGE GRADES: Good solid job on explanation, with 
excellent support from examples, data, or figures; excellent reasoning and 
explanations in the discussion, goes beyond the minimum required for the 
assignment. Writing has good organization, interesting sentences, few 
grammatical or spelling errors. DOES NOT read like a first draft. 

CRITERIA FOR "C" RANGE GRADES: Good solid job, does what the 
assignment asks, decent reasoning and explanations in the discussion, decent 
support from examples, data and figures. Writing has decent organization, 
serviceable prose, READS LIKE A FIRST DRAFT. Papers with "A" or "B" level 
writing but numerous errors in spelling and grammar (indicating lack of 
proofreading) will only receive a "C." 

CRITERIA FOR "D" RANGE GRADES: OK explanations and discussions but 
too general or some inaccuracies or flaws in reasoning or science; coverage is 
accurate but cursory and does not meet the minimum required for a complete 
answer. Writing is disorganized, awkward sentence structure; poor grammar and 
spelling; READS LIKE A FIRST DRAFT. 

CRITERIA FOR AN "F" GRADE: Doesn't effectively address the question, fails 
to support assertions with data or examples, unclear explanations, inadequate 
understanding demonstrated; major flaws in reasoning or explanations. Writing is 
very disorganized, awkward sentence structure makes it difficult to read; poor 
grammar and spelling. 



SOME EXAMPLES 

DISCUSSION GENERATING QUESTIONS 

In keeping with the authenticity of the evaluation, I encourage students to engage each 
other in debate via the class listserve. The requirement is a minimum of three posts per 
unit for 20% of the grade, but this minimum is far exceeded by the most active students. 
Students are asked to engage in lively debate, and follow common sense rules of good 
taste, manners, politeness, and netiquette. Coihmunicating, working together, debating, 
learning to ask questions - in many ways, the Internet and listserve (or any type of chat 
capability) provides richer opportunities for discussion than a face to face class. 
Comments and responses to questions are more thoughtful, less "off the cuff and likely 
more scientifically accurate. Students have less of an ability to "lurk," they are either 
actively participating or it is very apparent that they are not. This gives me the 
opportimity to intervene early enough to save a grade or advise the student drop the class. 
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One of the first discussion generators I pose is deceptively simple, giving me plenty of 
on-line teachable moments. I ask them to: 

"examine an accurate map of North America. List the state (considering all 50) 
that is: (a) farthest north, (b) farthest south, (c) farthest east, and (d) farthest 
west." 

Most students answer correctly that Alaska is farthest north and Hawaii is farthest south, 
but answers to the other two vary widely. Most do not guess right off that Alaska is 
simultaneously farthest east and west, as well as north. This leads into good discussions 
on the international dateline, both as observed’ and official, the meanings of "east" and 
"west," and the derivation of the geographic (latitude and longitude) grid system. 

Later discussion generators are not so simple. The following question, for example, ties 
several topics together, using a regionally important example: 

"Find as much information as you can about Yellowstone Park volcanic area, 
beginning with this web site: 

(http://vulcan.wr.usgs. govA^ olcanoes/Y ellowstone/firamework.html). This 
volcanic area is far from any present-day plate boundary, yet volcanic activity 
here is related to plate tectonics. Discuss the origin of this volcanic area, what the 
magma source is, and what the potential hazards are to people and the local 
environment. Think about this point: Yellowstone Park is a "repository" of sorts 
for wildlife - numerous species thrive here in great numbers, including some that 
are on the endangered list. How wise is it of us as a society to place these animals 
here in an ostensibly "protected" place, when at any moment, the volcano could 
come to life and wipe them all out?" 

This discussion encompasses aspects of the biosphere (biology, zoology), lithosphere 
(geology, volcanology), and human interactions with these environmental realms. It is 
open ended ("find as much information as you can...") which leads students in different 
directions and to various depths into the topic. When they come together in the listserve 
discussion phase, a rich information-sharing and lively discussion ensues. Students taking 
a science course for a general studies requirement will undoubtedly face a time where, in 
their professional lives or as a voting citizen, they must make a decision regarding issues 
of human interaction with the natural environment. 



MULTIPLE CHOICE QUESTIONS 

Even though I use multiple choice items for some of my assessment (30% of the possible 
points) and multiple choice is not necessarily authentic, I do so for these reasons. First, I 
use them to emphasize some terms and concepts that think are important and that I want 
students to pay extra attention to. If they re-read the appropriate section of the on-line or 
paper text material while answering the question, so much the better; for some, sad to 
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say, it may be the only time they read the text. The idea is to let them know what is 
important, and this is a familiar way to do so. • 

Some of my questions are rather ordinary, if they don't do well at these, they will not do 
well in the class, and if they find out early enough students can get into another class or 
drop this one. For example, the question: 

"The angular inclination of the Earth's axis with respect to the plane of the ecliptic 
A) varies through the year B) is 0 degrees C) is 23.5 degrees D) is 66.5 degrees E) 
is 90 degrees." 

This question illustrates a basic point, and seeking the correct answer leads to several 
other basic points. The Earth's axis does not vary much from 23.5 degrees tilt away from 
the perpendicular to the plane of the ecliptic. This enters into concepts of seasonality, 
climate, climate change, and so on. If a student cannot be bothered to look up and read 
this material, it is best that we both find out about it early. A follow-up question is a bit 
more difficult; 

"The fact that at any time diuing the year, the earth's axis is parallel to its 
orientation at all other times is called its parallelism, or: A) revolution B) rotation 
C) polarity D) aphelion E) perihelion." 

This is nearly the same question; it deals with the same issue but a different aspect of it. 
The polarity of the Earth's axis means that it is always pointed to the same spot (the North 
Star) all of the time. This again emphasizes the concepts of seasonality, climate, and 
climate change. 

Others of my multiple-choice questions are not ordinary. For example: 

"You are navigator on a ship in the South Seas. You have traveled for 6 days due 
south. You have noticed that the sun, at 12:00 noon, is now about 12 degrees 
lower in the sky. Estimate about how far south have you traveled. A. 200 km., B. 
666 km., C. 1332 km., D. 828 km., E. cannot tell with information given." 

More than half of my students answer E, "cannot tell with information given" because 
they do not stop and take the time (despite the quiz being on-line and accessible 24 hours 
of the day) to think about what is being said. Due south must be along a great circle, each 
degree traveled along a great circle covers about 111 kilometers, so the correct answer 
must be 1332 kilometers. The answer to this question cannot be looked up in a book, it 
can only be calculated by putting together several disparate factual items, and calculating 
the result. 

Another is a favorite of mine because I hear rumors (unsubstantiated) that it is a question 
asked of graduating seniors at an ivy league school, most of whom could not answer it 
correctly: 
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"The best reason to explain why winter is usually colder than summer is: A. The 
sun is lower in the sky and the days are shorter, B. The snow reflects incoming 
solar radiation, C. There are more clouds in the winter to intercept solar radiation, 
D. The earth is farthest from the sun in the winter, E. The sun radiates less in the 
winter. 

Surprisingly, most people answer "farthest from the sun" when in fact the answer is "the 
sun is lower in the sky and the days are shorter." This is a thinking question, and again 
the answer can only be found by putting together two or more bits of information. 



ESSAY QUESTIONS 

Open-ended essay questions are the easiest and quickest to write, and the most difficult to 
evaluate. Even though they are straightforward as far as the topics go, they are not easy 
for a student to do well at. This is a very authentic experience in that many things we do 
in our professional lives are rather mundane - mundane but important, and we need to be 
able to effectively communicate the ordinary results and insights in as clear a language as 
the most profound. Open-ended questions give the students the opportunity to 
demonstrate mastery over written communication and understanding of the science. One 
of my first questions is to: 

"Explain in your own words the differences between a large scale map and a 
small scale map." 

I do not accept ordinary answers, written straight from the book. Students have a copy of 
the scoring rubric, I follow it faithfully, and I look for signs of "unusual insight" or 
"flashes of brilliance." Many of the open-ended essay questions are the typical compare, 
contrast, and discuss type of questions. For example: 

"Describe, discuss and compare the four mechanisms that can result in the uplift, 
cooling, and ultimately condensation of an air mass." 

There is nothing surprising here, the four mechanisms can be looked up in any textbook. 
The student (very authentically) is not being evaluated on the memorization of four items, 
but on the ability to communicate his or her understanding of these four mechanisms in a 
clear maimer. 



AUTHENTIC PROJECTS 

I consider laboratory projects to be the most important part of Internet science courses. 
Written (and illustrated) reports of project results count for 50% of the final grade. The 
Internet medium lends itself well to a laboratory setting, but you may have to shed some 
old ideas of what "LAB" really is! In traditional geography labs, students would be 
working with and answering questions about maps and other images. It is no different in 
an Internet class, except that the images come from the Internet. 
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Laboratory projects allow the student to get closer to the real world and real world 
("authentic") problem solving. It also is a good training ground to obtain an appreciation 
of scientific methods and develop communication skills. Good information, insightful 
ideas, well developed solutions to problems, and brilliant analyses are all useless unless 
students are able to communicate them to others. Students are required to work in teams 
for two out of the eight projects, and encouraged to work together on the rest. The 
requirement that they work together causes more complaints than anything else we do in 
Internet classes, even though most students recognize the necessity of learning to do 
group work. The biggest fear is, of course, that their hard work will go unrecognized, and 
that others will benefit fi'om their efforts. Nothing could be more "authentic" in student 
training for the real world! It is relatively easy to determine in short order who 
participates and who does not, making such fears groundless. Students are held 
individually responsible for knowing the material, and they are responsible for their own 
reports. 

There is some overlap between projects and discussion questions, and this is intentional. 
Projects differ in that students are required to be more thorough in their research, more 
complete in their reports, and they are given more time to complete the assigrunents. 
Projects require work with the textbook, the instructor, the Internet, and other outside 
sources. Each project is designed to explore a key concept, a body of knowledge, or a 
method of study and analysis. 

The first project is straightforward and relatively simple, designed to introduce the 
students to the resources on the web, how to access those resources, and how to 
communicate the results their projects to the instructor and their classmates. I ask for a 
simple on-line search: 

"After you read the chapters and visit this site, do a search on the topic of "Earth 
Systems Science," "Physical Geography," and/or "Environmental Science." Send 
the results of your search to me, and I will collate the results and share them with 
the rest of the class. Please armotate your search results, that is, tell the rest of the 
class something about the sites, are they good and scientific, or off the wall? You 
will find all kinds on the net! Are they useful? Does they lead to other useful 
sites? Are they "user friendly?" 

This project has a realistic goal; one that can be accomplished in short order, and one that 
gets students familiar with the mechanics of the Internet faster than most anything that an 
instructor can assign. It is important to facilitate early success in what might be a 
daunting medium for many students. I have seen students absolutely blossom and "take 
off after this project, like opening the floodgates to all the knowledge of the world. 

The second activity is not so much fun. Students are assigned to groups, a group leader is 
selected, and a group project assigned. I do not intervene in the group dynamics unless it 
is an emergency. I leave it up to the students to divide the labor, set their own deadlines 
(within my time frame) and decide the final format. I encourage electronic submittals, but 
I accept any format except handwritten. Some groups will "carry" a non-participant, and 
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others will complain loudly and at length about a similar non-participant. The group 
project is based on another search: 

"Your group project for this two- week period is to search the internet on the 
topics of 1) atmosphere composition, energy and temperature, and structure and 
2) ozone in the atmosphere. The first is a general topic, and the second is a 
specific topic within the first topic. You will likely find lots of information on 
ozone, and perhaps not so much on the general composition, temperature and 
structure. However, do not ignore one topic at the expense of the other. The ozone 
problem cannot be fully understood without understanding its atmospheric 
context! Summarize and analyze your search results: Write a short report that 
demonstrates your understanding of atmospheric composition, energy, and 
structme. Specifically address the ozone layer and the apparent ozone problem. 
Your report should contain: 1) introduction, 2) summary of background 
information on the composition, structure, and energy content of the atmosphere, 
appropriately referenced, 3) a discussion of ozone and the ozone problem, and 4) 
a summary and wrap-up section. You should clearly distinguish your group's 
thoughts from your sources. It is strongly suggested that your report be 
appropriately illustrated. Please contact me with any problems well in advance of 
the deadline so that I can help you!" 

This group project, which comes prior to the midterm, helps the students not only work 
as a productive team members, but it also helps them become familiar with downloading 
and submitting various digital formats for both pictures and text. It does not matter what 
the specific content area is in this case, it is very likely that these students will be 
involved with these tasks at some point in their education or their careers. 

One of my favorite realistic projects deals with determining the return interval for floods 
of various sizes on most rivers in the United States. Not many students successfully finish 
this project; it is difficult, involving mathematics, spreadsheet analysis, graphing, and 
scientific interpretation. This is about as authentic as it comes - sometimes in the 
workplace we do not successfully finish a project, and they most often involve 
spreadsheets and computations. 

Students download archived records of the peak flows for the Blackfoot River in 
Montana. The records run from 1940 to 1994, and some of the records are not necessary 
and not used, another authentic touch - how do we separate the data we need from the 
data we do not. Following instructions, students open the downloaded data into a 
spreadsheet, perform the mathematical analysis, and graph the final results. This 
generates a great deal of discussion, very high level for those with good mathematical 
background, and somewhat ordinary for those with poorer backgrounds. However, none 
of the analysis is obscure or difficult to follow, and nearly anyone can perform the 
analysis. Most of the questions involve "how to" manipulate data in a spreadsheet, open 
data files, download, and so on. After the analysis of the Blackfoot River is finished, 
students are asked to chose their own "favorite" river, and do a similar analysis. Most 
pick one close to their hometown, and some have been so enamored in the project they 
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have interviewed local flood insurance salesmen and county planners. Again, this is the 
type of issue that can come up in profession or personal lives of students. This type of 
authentic assessment project gives them a background to understand complex issues. 



CONCLUSIONS 

On-line assessment of small web-based classes in small schools can be accomplished 
using many of the same assessment tools used in face-to-face classes. Student’s 
evaluations should be weighted towards authentic assessments, assessments that require 
students to demonstrate skills sought after by today’s workplace: communications skills, 
problem solving ability, and teamwork. Many of the assessment college instructors are 
already doing can be adapted to the Internet environment. 



NOTES 

[1] Secretary's Commission on Achieving Necessary Skills. What Work Requires of Schools. Washington, 
DC: SCANS, U.S. Department of Labor, 1991. 

[2] A complete listing of my Internet classes, including Earth Systems Science approach to Physical 
Geography, can be found at this URL: http://wind.cc.whecn.edu/--gnelson/claslist.html. This site is not 
password protected, and also includes links to my online Montana State University Class in Earth Systems 
Science for K-14 educators, and my on-line University of Wyoming Class in Cultural Ecology. All 
examples in this paper come from this site. 

[3] Definitions from the Thesaums of ERIC Descriptors and the ERIC Identifier Authority List pertaining 
to concepts related to Authentic, Alternative, and Performance Based Assessment, ERIC Clearinghouse on 
Assessment and Evaluation (http://ericae.net), Shriver Laboratory, College Park, MD 20742. 

[4] Wiggins, Grant, 1990, The Case for Authentic Assessment, ERIC Digest, No. ED32861 1 

[5] Tewksbury, B.J., 1996, teaching without exams: The challenges and benefits. Journal of Geoscience 
Education, p. 366-372. 
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